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ABSTRACT 

Task of an investigiition coordinated by a conaittee 
working under the auspices of the National Council for Social Studies 
describes the nodel used by the National Assessment for dat^ 
gathering and reporting on the citizenship area and discusse.^ its 
potential uses for state and local assessment, curriculum 
development, and accountability purposes. The seven basic components 
of the assessment model include objectives development, exercise 
development, sampling plan, administration of exercises, scoring and 
analysis, reporting and dissemination, and utilization of 
information. Maine, Connecticut, Texas, and Colorado have adopted the 
national model for use in state assesr^uents of Ivsarning outcomes. To 
date three school districts— Lincoln, Nebraska, San Bernardino, 
California, and Montgomery County, Maryland— are using an adaptation 
of the model at the district level. Thirteen characteristics for a 
good assessment program are suggested for developing an adaptation 
model. The reports on both science and citizenship have resulted in 
strong recommendations for curriculum changes. As data are gathered 
at the state level through the use of the model and its adaptations, 
specific suggestions for changes may be made. Based on the national 
model, adaptations are proposed which provide a basis for both 
teacher and program accountability. <DE) 
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POTENTIAL USES OP THE NATIONAL ASSESSMENT MODEL 
AT THE STATE AND LOCAL LEVELS 



by 

Bob L. Taylor 
University of Colorado 

Under National Assessment, several volumes of data have al* 
n:>eady been collected and published. There no longer Is a debate 
concerning the pros and cons of having a National Assessment pro* 
in^am, for the results of the early assessments are now available 
i;o "the public. Educators need to study and Interpret the results 
sa^rallable as a base for decision making In education* Along with 
:thls, they need to understand how these data were collected and 
iWhat were the problems and limitations of the data gathering process. 

'National Assessment Is a plan for the systematic, census^llke 
saurvey of knowledges, skills, understandings, and attitudes. It 
its m Information gathering plan aimed at providing both educators 
and the lay public with Information concerning the level of achieve- 
tssent In selected subject areas for students and young adults. The 
®tMil :1s to provide information that will be used to Improve educa- 
±ilxm.. It Is concerned with the achievement status of four age levels 
jin tten different subject areas. The subject areas selected for 
assessment were: Art, Career and Occupational Development, Citizen- 
sahlpk, ^Literature, Mathematics, Music, Reading, Science, Social 
SBabJttdies, and Writing. The basic task of this paper U to describe 
tthe ^nrodel used by National Assessment for data saoherlng and re- 
porting on the Citizenship ar€^a ,,o discuss the potential uses 



of the model for state and local assessment, curriculum develop- 
nenti and accountability purposes. It Is not within the scope of 
this paper to criticize the model with respect to technical flaws; 
hence, the model Is described and discussed without reference to 
any potential problems of design. 

THE MODEL 

The model Is in the continuous process of being refined and 
improved, thus only the basic components of the model are presented 
in Diagram !• A circular scheme is used in presenting it since, 
in reality, its actual application may be Initiated with any one 
of the components. Also, in its actual application, there are con" 
tinual interactions between and among the various components. While 
theoretically the process starts with the refinement of overall 
national goals into specific subject matter, behavioral objectives, 
and profjresses in logical sequence through to the final Utiliza- 
tion of the Information, in practice there is much greater freedom 
with respect to the utilization of the components. 

The model for the Citizenship Assessment is presented here in 
outline form with a fairly detailed description of its components. 
As presented in Diagram I, there are seven basic components Identi- 
fied in the models Objectives Development, Exercises Development, 
Sampling Plan, Administration of Exercises, Scoring and Analysis, 
Heportlng and Dissemination, and Utilization of Information. V/hlle 
many of the fine points of the model are not developed in the fol- 
lowing outline, it is described in sufficient detail to give the 
reader a good understanding of how the data were collected and what 
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Implications might result from these data. The number of sub* 
topics In the model and their distribution Indicate that the major 
efforts of National Assessment have been with the first five com* 
ponents* The last three components have been areas of controversy 
and, therefore, have received less attention until recently. 



Outline Assessment Model for Citizenship ^ 

!• Objectives Development^ ♦3, ^,5 

A* The task of developing objectives In the field of 

citizenship was awarded to the American Institute for 
Research of Palo Alto, California, These criteria 
were used in examining the objectives: 

(a) They were considered Important by scholars. 

(b) They were accepted as an educational task by the 
school . 

(c) They were considered desirable by thoughtful lay 
citizens* 

Scholars reviewed the objectives for authenticity with 
respect to their subject fields; school people reviewed 
the objectives in terms of their actual emphasis in 
their schools; and laymen reviewed them in terms of 
their experiences with regard to their value in life. 

B, The American Institute for Research staff reviewed pre- 
vious lists of citizenship objectives and boiled these 
down to one comprehensive list of 20 objectives. 



•""Womer, Frank 3., >/hat Is National Assessment ? National Assess- 
ment of Educational Progress, Denver, Colo,, 1970, 

Morris, Eleanor L, (Ed.), Citizenship O bjectives , Committee on 
Assessing the Progress of Education, Ann Arbor, Mich,, 1969, 

3campbell, Vincent N., £i al» Citizenship Objectives 122^-25 
Assessment , Education Commission of the States. National 
Assessment of Educational Progress, Denver, Colo,, 1972, 

Campbell, Vincent N. and Daryl G, Nichols, "National Assessment 
of Citizenship Education," Social Educratlon 32:279-81, June, 
1969, 

^Campbell, Vincent N,, sJ^ sJ,. Repor . t 2, Citizenshlp t Natio nal 
Results ^ Education Commission of the states, National 
Assessment of Educational Progress, Denver, Colo,, November, 
1970, 
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C. Outstanding local teachers familiar with each target- 
age group (9, 13» 17, adult) , working with the American 
Institute for Research staff, broke down each general 
objective into the most germane behaviors deemed appro- 
priate as goals for a given age group. 

-D* A selected group of students and adults In each age 
group was asked by the American Institute for Research 
staff to recall and describe outstanding citizens of 
their acquaintance and specific Incidents reflecting 
good and poor citizenship. These incidents and deccrip- 
tions, about 1,000. were used to check the completeness 
of the initial list of objectives. 

-B. The objectives were stated on three levels (general 

objectives, sub-objectives, and behavioral age illustra- 
tions or statements). The results were summarized for 
each age group. 

"P. The revised list of objectives, broken down into im- 
portant behaviors, was then worked over for three days 
by a panel of national leaders in citizenship education 
and related social sciences. 

:G. A group of persons in various roles from selected 

California communities reviewed the objectives and made 
suggestions. These Included public and private school 
administrators, counselors, teachers, a judge, a county 
planner, labor and business leaders, and social scientists # 

Ifi. The objectives were then reviewed by panels of laymen. 
Eleven lay review panels representing four geographic 
areas of the country and three different community sizes 
were used. Each panel spent two days reviewing the 
objectives based on these two questions: "Is this some- 
thing Important for people to learn today?" and "Is 
this something I would like to have try children learn?" 

HI. TExerclses Development^ »^»^ 



"^Vforaer, clt . 

Roadway, Charles J. (Ed.) Re ading and Lite rature * General 

Information Yearbook , Education Commission of the States, 
Report 02-Giy, National Assessment of Educational Progress, 
Denver, Colo., May, 1972. 

^?Elhley, Carmen J. and Frances r>. Berdle, The National Assess - 
>ment Anproaoh Exercise De velopment ^ National Assessment 
'Of Bducatlocal Progress, Ann Arbor, Mich., 1970. 
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A» The production of the exercises was Initiated by the 
American Institute for Research in 1966, The exercises 
were developed to cover all of the major objectives and 
to represent the selected content areas* Many exercises 
required the use of interview techniques, as well as the 
usual pencil and paper exercises. Also, self report and 
group task exercises were used. 

B* Because National Assessment intends to describe what 
people in an age group know, the exercises were written 
to reflect three difficulty levels—reporting knowledge 
or skills common to almost all persons in an age group, 
reporting skills or understandings of a typical member 
of an age group, and reporting understandings or knowledge 
developed by the most able persons In an age group. 

C. All exercises were developed to meet these criterias 
content validity, clarity, functional exercise format, 
clustering exercises based on a single set of stimulus 
materials, directionality of response, difficulty level, 
content sampling, and overlap between age groups. The 
exercises were direct measures of some pieces of knowledge 
understandings, attitudes, or skills which were mentioned 
in one or more of the objectives. 

D; The exercises. were reviewed by panels of lay persons for 
clarity, meaningfulness, and invasion of privacy. 

E« There was a tryout of the exercises involving repre- 
sentatives of groups in the actual assessment— regions, 
communities, races, sexes, and age groups. Following 
the tryouts, the American Institute for Research staff 
and subject matter specialists reviewed the tryout data 
and made needed revisions. 

P;. A committee of subject matter specialists, measurement 
specialists, and National Assessment staff members rated 
the exercises to be included in the packages according 
to a set of criteria, and based on the ratings the exer- 
cises were selected. 

G;. The selected exercises were reviewed by U. S. Office of 
Education personnel for any infringement of privacy on 
the part of the respondents or possible offensiveness. 

Hi. Since there were about 160 minutes of testing time avail- 
able for each age group in each subject area, the exer- 
cises used were only a small sample of the potential 
number of exercises. The exercises were assembled into 
administrative units (packages) for groups up to 12 
persons. 
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m. Sampling Plan"^'^^ 

A. The sampllriK plan was sutcontracted to Research Triangle 
Institute, Raleigh, North Carolina. A multi-stage design 
was used which was stratified by region, size of community, 
and socio-economic status. This was a probability sample 
which allowed researchers to collect data from a small 
sample of the population and to infer from that sample 
certain characteristics of the entire population. 

B. The populations for assessment were .all 9 year olds, 
all 13 year olds, all 17 year olds, and all young adults 
26 through 35 years old in the 50 States plus the District 
of Columbia. The only exceptions were the exclusions of 
institutionalized individuals of these ages— those in 
hospitals, prisons, and others who could not be reached. 

C. For ages ^ and 13, a school sample only was used and for 
the 26 through 35 age group a household sample only was 
used. For the 17 year olds, both a school and a house- 
hold sample were used. 

D. The entire country was divided into population areas as 
follows: cities, counties exclusive of cities, and 
pseudo-counties— two or more counties were put together 
when the population of a single county was less than 
16,000. Bach population unit of 16,000 residents was 
assigned a number. 

B. The country also was divided into four geographic 
regions: Northeast, Southeast, Central, and West. 

P. Each geographic region was divided into communities of 
four types: large cities of above 180,000 population, 
urban fringe, middle-sized cities between 25*^000 to 
180,000 population, and small town-rural of under 25,000 
population. 

0. The 52 sampling units for each geographic area were 

spread across the four community types in a fashion pro- 
portional to their population in relation to the area 
population. 

H. To Insure comparable representation from each part of 
the country, an equal number of sampling units was 
selected from each geographic region— 52 from each of 
the four regions for a total of 208. 

^Morris, Eleanor L.. e^ al, ^ejiQT^ 1, 1Q6Q - 1970 S cience : N^.tlot^Al 
Results aai Illustrations o£ O , roup Comparisons , J. R. Chrony 
and D. G. Horvitz, "Structure of Sampling and Weighting." 
Appendix C, Education Commission of the States, National 
Assessment of Educational Proe;ress, Denver, Colo., July, 1970. 

lOKorris, Citizenship ObleotlveSy qq. sife. 
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i; The actual sampling units chosen were selected at 

random. This plan did not guarantee that all 50 States 
would be included in the sample. This was not a survey 
objective, but later the design was changed so each 
state was included in the sample. 

Jr In each sampling unit selected, all school buildings 
enrolling students of the sample ages (public, private, 
and parochial) were identified. 

K. The plan for schools was to select units of approxi«iiately 
250 to 350 pupils for each age group and from at least 
two different buildings within each sampling unit for 
each age group. 

L.. Each cooperating building principal provided a list 'of 
names of students in the building from the specific ag*i 
groups. This list was used for the final random selection 
of students to take the assessment exercises frow that 
building. 

M. Information about the areas was obtained from the S. 
census data. In order to report reliable information 
for lower socio-economic status groups, these groups 
were oversampled. There was a disproportionate number 
of schools from lower socio-economic status areas in- 
cluded. In the overall results, the data from the lower 
socio-economic areas were given the percentage value in 
which ichey occurred in the total population. 

N. Prom each of the 20B geographical samples, 100 adults, 
ages 26 through 35, were randomly selected using the fol- 
lowing procedures. Each of the 208 geographic samples 
was divided into eaual secondary sampling units. Then 
ten secondary sampling units were randomly selected from 
the totil 20fi samples. Interviewers then personally 
contacted the people in the chosen secondary sampling 
units of the 26 through 35 age group and out-of-sehool 
17 year olds. These persons were asked to participate 
in the assessment. 

0^ Individuals were classified as black, v^ite, and other 
on the basis of information provided by the school or 
by observation. Results were given for black and white 
only. The number of individuals classified as other 
was too small to produce reliable results. 

IV;\ Administration of Bxercises"^''''^^ 



^^dadway, 



A. The administration of the exercises was subcontracted 
to Research Triangle Institute in the East and to 
Measurement Research Center of Westinghouse Learning 
Corporation, Iowa City, Iowa, in the West. Cooperation 
of schools was obtained by first contacting officials 
at the state and then at school district levels. There 
was above 90 per cent cooperation by schools. 

Adults and out-of-school 17 year olds were contacted 
by a personal door-to-door household canvass. Each 
out-of-school partic leant was contacted individually. 
All had the right to refuse to cooperate, 

B. A full-time trained staff of 27 district supervisors 
managed the field work. They were assigned to different 
geographical areas of the United States. They contacted 
schools and recruited and trained local teachers to 
help in the admlnls oration of the exercises in schools 
and recruited and trained other available persons for 
tiVe owt-of-school administration. 

i 

C. la the schools, students from a slnpjle age group from 
different classes were brought tocjether in a room for 
exercise administration. Group size was at least 8 
and usually 12 students. 

D. The exercises were organised in packages which contained 
exercises from two or three different subject areas at 

a single age level. Nd one person took all the exercises 
in his age group. Age groups were assessed at different 
times of~ the year. 

E. In packages administered to groups, taped directions and 
taped readings of the exercises were used in addition 

to printed packages. This was done to establish con- 
sistency in" timing and administration plus to provide 
for non readers. 

P. Several packages at ages 9, 13, and 17 conslsted-of exer- 
cises that v/ere given by exercise administrators to one 
Individual at a time. The administration of all the 
packages for the adult assessment was done by interviews. 

G, Bach package required about ?0 minutes of administrative 
time. Each person took only one package with the ex- ^ 
ceptlon of the out-of-school, 17 year olds who were askea 
to take four or five packages each since they were the 
most difficult and expensive group to locate, 

H. Students* names were confidential and did not appear 

on any packages. The name roster was kept at the build- 
ing level and used only in the organljsation of the in- 
school sampling. 

V, Scoring and Analysis ^^^^ 



X^Sadway, xiB# ail. 



10 



A. The scoring and analysis of the exercises were sub- 
contracted to Measurement Research Center of Westlrtghouse 
Iiearnlng Corporation, Iowa City, Iowa. 

B. The multiple-choice exercises were scored and recorded 
routinely by i^achine. 

C. The openended exercises were scored by trained profes- 
sionals using a key of acceptable and unacceptable 
achievements in terms of the objectives. 

D* Results were reported for each goal. Also, the results 
were reported both as the pereentaRe of any group of 
respondents making the desired responses to an exercise 
&nd as the difference between the percentage of a group 
making the desired responses and the corresponding 
national percentage. 

E. In the assessment, there was a lack of proportionality 
among characteristics used in the comparison of groups, 
such as color, sex, parental education. A statistical 
procedure, balancing, was used to correct for this problem 
in the comparative analysis of the data. Balancing is 
a procedure to examine the performance of groups classi- 
fied on one characteristic adjusting for the fact that 
these groups differ on a specified set of other char- 
acteristics. 

VI. Reporting and Dissemination^^»^^»^'' 

At The reporting of results was directed to subject matter 
specialists, professional educators, and informed laymen. 
Multiple reports were developed to serve these different 
audience s» 

B. Approximately ^0 per cent of the exercises were reported 
at the end of each assessment year. Not all exercises 
were reported since they were to be used over again in 
future assessments in order to measure change by means 

of comparing the results on the uncontaminated exercises. 

C, The exercises released for publication were selected to 

be representf»tive of all exercises administered as well as 
the results received on the assessment. 



'^''Womer, ,cit . 

^^Campbell, Report 2, Citizenship t National Results . filjfe. 
17Gadway, jsfi. 



11 

D. Reporting was done by 9, 13 » 17, and 26 through 35 age 
groups. Since the same exercises were used with dif- 
fei'ent age groups, there was comparable data across two 
or more age levels. 

B. Reporting was also done by groups within the categories 
of regions, community types, sex, socio-economic status, 
and white, black, and other. 

■ P. Pinal reports v/ere printed with a short description of 
the exercises, the national percentage of success, and 
group differences from the national percentage of success 
for each exercise. This was done without any interpre- 
tation of results. 

. G. Both observed and balanced results for all exercises 

and by groups wt're reported. The effects of balancing 
on measured characteristics such as sex and region were 
included in the report. 

. H. There were no scores reported for individuals. No 
single Individual took more than one twelfth of the 
exercises, and no individual took a package that sampled 
a single sub.lect area. 

i I. Results wej^e reported through the media: written word, 
radio, television, films, and personal reports. 

•TVII.. Utilization of Information^^ ♦^^ 

. A. The results provided information for educational decision 
making. For example, considering the somewhat lower 
performance of the Southeast Region on the Citizenship 
results, school boards In that region might decide to 
put greater stress in their school programs on citizen- 
ship skills, understandings, and attitudes. 20 



^•p- ' ' 

v*-"Womer, clt . 

'■^*3*%onaway, Larry B,, "Some Implications of the National Assessment 
;. Model and Data for State and Local Education," Paper Presented 
t-.at the 1973 Annual Meeting of the AERA, New Orleans, February 
^ 26, 1973. 

2''RPcampbeU, Vincent N., Manford J* Ferris, and Daryl 0. Nichols, 
. • National Assessment R eport fi, , 196.9 - 1Q70 Cltlzenshla t Grou p 
Results fpr rjex, Regi . on , , and ^Ize of Commqaltv , Education 
Commission of the States. National Assessment of Educational 
V- Progress, Denver, July, 1971. 
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B. The results raised many questions which may lead to 
other Investigations. For example, In making comparisons 
of all Citizenship results combined, It was found that 
the Extreme Affluent Suburbs showed substantial median 
advantages at all ages and that the Extreme Rural and 
Extreme Inner City showed substantial deficits at h;11 
ages. Here are discrepancies In performance which need 
to have causal studies conducted on them from the per- 
spectives of different disciplines such as political 
science, sociology, economics, and education.^l 

A second example is that assessment could expand Into 
other educational areas, such as assess a sample of 5 
year olds for a basis of comparison, break results down 
by states, or expand into new subject areas* 

C. The results of several cycles should provide evidence 
of the change in knowledge, skills, understandings, and 
attitudes in the age groups as they relate to educational 
objectives. 

D. School administrators can make comparisons between 
groups, and may Improve student performance from the 
information gained in this manner. 

Prom this review of the model, it is evident that the National 
Assessment staff has put a great deal of effort and know-how into 
the design, plus the development of each of the components. Each 
of the components reflects the best available information in the 
field of assessment. 

A .complicated but logical process has evolved for identifying 
the behavioral objectives for the specific subject areas. The use 
of laymen along with teachers and subject-matter specialists added 
validity checks on these objectives. Considerable inventive talent 
vas displayed in writing the exercises for assessing some of the non- 
cognitive type of objectives. The use of the tape-paced administration 



IJorrls, Eleanor L., Vincent N. Campbell, Manford J. Ferris, and 
Carmen J* Flnley, Rational A ssessment Repo rt ^. ,19,6 9-, !, 97 , 0 
CltlzenshiP i Group Results for P arental Education , Color ^ 
, Size , a nd T ype, ^ Community , Education Commission of the 
States, National Assessment of Educational Progress, Denver, 
May, 1972. 
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was a valuable innovation in the field of testing. In this -pro- 
cedure, a complete set of directions for the administration of a 
package of exercises is recorded on an audio tape 4 By playing 
the tape, the directions are always given in the same manner and 
the exercises are timed. Also, the problem of nonreaders is over- 
come through the oral directions. While the technology had bsen 
available prior to this use. It had not been widely used. The pro- 
cedures for scoring subjective exercises have provided a model 
for accurate, objective scoring of these kinds of exercises. The 
scorers were put through a special training program for scoring 
essays, written materials, or performance exercises. 

The results ha'/e been released in census-like fashion without 
Interpretation. Clear, factual reports were made of the data so 
the reader would know what was assessed and what were the results. 
Very little interpretation of the results has been done for the 
reader. This was the Intent of the Natirnal Assessment staff, for 
they did not conceive of their role as going beyond the reporting 
function. 

The final two components of the model are the most contro- 
versial, for originally there was not a clear mandate for National 
Assessment to tak? a vigorous leadership role with respect to these 
two tasks. As time has gone by, greater activity by National 
Assessment in these areas has been urged by such groups as edu- 
cators and parents, and National Assessment has become more active 
in the dissemination and utilization components. 

In summary, it can be pointed out that the model reflects 
some important choices on the part of the staff of National 

ERIC 
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Assessment. %e decision was made to assess a broad range of ob« 
jectlves in each of the ten subject-matter fields. Certainly, It 
would have been much easier and cheaper to have concentrated on 
a mirrow set of objectives. Also, the effort was successfully 
made to include the higher cognitive levels in the assessment 
exercises and to deal with the affective, domain.. In subject areas 
like Citizenship and Social Studies, the usual standardized test 
concentrates on factual knowledge which is of a less controversial 
nature and easier to assess* National Assessment should be com-* 
mended for their bolder, more comprehensive approach to the task 
which searches for more significant kinds of data* 

Changes iq ,the Model 

From the time the assessment in Citizenship was carried out 
until the Social Studies assessment was conducted, there were 
changes made in the assessment model* This can be Illustrated by 
the -changes made in the procedures for identifying objectives* 
For the Citizenship objectives, the American Institute for Research 
at Palo Alto, California was the contractor, while for the Social 
Studies objectives, the contractor was Educational Testing Ser- 
vices-Princeton, New Jersey. Referring to the Objective Section 
of the Outline of the Model, you can compare the process used in 
establishing the Citizenship objectives with those presented here 
for ^establishing the objectives for the Social Studies, 
22 ■ 

Norrls, Kleanor L. and Barbara Goodwin, (Eds,), Sgclal Studies 
Objectives , Education Commission of the States, National 
Assessment of Educational Progress, Denver, Colo*, 1970* 
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The following were the steps in the procedure used in 
establishing the Social Studies objectives: 

1,. The task of developing objectives in the Social Studies 
was awarded to Educational Testing Service, Princeton, New Jersey • 

2* A committee of 11 social scientists plus Educational 
Testing Service staff members prepared an initial draft of the 
objectives for Social Studies, 

3. Prom the nominations by officers of various national and 
state organizations in education, persons were selected to form 
11 lay-review panels. These individuals represented rural, small 
town, suburban, and large cities in the four geographic regions of • 
the United States, 

If. A social studies conference of 21 people including scholars 
from the fields that contribute to the area of the social studies, 
leading social studies teachers, university professors who special- 
ize in social studies education, and representatives of research 
organizations with potential for reworking the objectives met to 
discuss the need for revision and for clarification of the existing 
statement of objectives, 

5. Following the design used for the development of the 
original objectives, the Educational Testing Service staff pre- 
pared a preliminary revision of the list of objectives, 

6. A committee of four social studies educators met and re- 
viewed the Educational Testing Service staff»s preliminary revision, 

7. The revised statement of objectives was submitted to ad- 
ditional reviews, one by subject matter specialists and the other 
by elementary and high school teachers, 

8. The objectives were then mailed for reactions to one mem- 
ber of each of the 11 lay panels which had reviewed the objectives 
earlier. The revised report was then accepted as the statement or 
objectives for the Social Studies assessment. 

There are important differences in how the Citizenship and 
Social Studies objectives were developed. To some extent, this 
reflects the respective characteristics of the two fields. In the 
development of the Citizenship objectives, greater stress was 
placed on input from laymen, students, and teachers. Also, while 
the social studies effort put more stress on academic representa- 
tion, it also went for more widespread geographic representation. 
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The Citizenship effort sought wider representation from different 
groups of people, but It was concentrated rather heavily In 
California* 

The operational details of the components will vary from one 
assessment to the next depending on such factors as what group Is 
awarded the contract for different parts of the assessment, what 
Is the subject field being assessed, or what academic scholars are 
Involved In the assessment. This flexibility makes for both 
strengths and weaknesses. A strength is that there are variations 
in the approach and old problems may be corrected. Weaknesses are 
that different assessment cycles may reflect varying philosophic 
positions both within the subject field and in the assessment pro- 
cess so that the data may not be strictly comparable. Also, it 
does mean that anyone studying assessment results must determine 
just how the data with which he is concerned were collected, for 
there may be important differences in the assessment process with 
which he must be familiar in order to interpret the data correctly. 

The above descriptions demonstrate that revisions in the 
assessment process are ongoing and that the model is constantly 
being changed with respect to the details of the components. V/hile 
it is possible to identify the major components of the model, it 
is not possible to give a precise, up-to-date description of the 
components because they are in the process of continual change. 
This is one reason why the Citizenship Assessment model was used 
in this paper, since it (with hindsight) could be reported in 
greater detail, and it does provide the reader with a good under- 
standing of how the National Assessment process operates. 

ERIC 
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National Assessment is a census-like study to collect Infor- 

.. matlon concerning the educational attainments of Americans, In 

...planning for the collection of this census-like data, the model, 

which was presented in the previous section, was developed. A 

r.:nomber of states have found adaptations of the model useful in 

: ^conducting state assessments in which desirable learning outcomes 

•are identified and the status of learners with respect to these 

--outcomes is determined. 

State assessment is a rapidly developing movement. At this 

.-writing, all of the states have assessment activities either in 

23 

r*-operatlon, in a developmental process, or in a planning stage, 
: While the statewide assessment programs have many similarities, 
•: .they break down into two basic types of programs on the question, 

"Who gets to use the results?" The divisions are those states for 
•v- which data are collected for decision making by state agencies and 
T'.those states for which data are collected for decision making by 
vfc teachers and administrators. State programs for which the emphasis 
i<ls on collecting information for state level decision making ares 
^.rArizona, California, Colorado, Connecticut, District of Columbia, 
rjJPlbrida, Maine, Massachusetts, Michigan, Nevada, New Jersey, New 
icYork, North Carolina, Rhode Island, South Carolina, Tennessee, and 
:cTexas, Programs for which the emphasis is on collecting information 

''■■?^State Bducal^ional Assessment Programs ^ 1^73 R evision ^ Joan S, 
•. Beers and Paul B. Campbell, "Statewide Educational Assess- 
tv.ment," Educational Testing Service, Princeton, New Jersey, 
ri973, p. 1. 
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for local level decision making are: Alabama, Arkansas, Delaware, 
Hawaii, Idaho, Iowa, Kentucky, Mississippi, New Hampshire, New 
York, North Dakota, New Mexico, •^nd Pennsylvania.*^^ 

Beers and Campbell report that a number of characteristics 
are appearing in these assessment programs. In about a third of 
the states, the programs were mandated by the state legislatures, 
and the results of the assessments are to be reported back to the 
state legislatures. In a few of the states, the data are to be 
used for PPBS (Planning, Programming, and Budgeting Systems). In 
about half of the states where the assessment data are being used 
to make state-level decisions, state and Federal funds will be 
allocated based on the results. Participation in assessment is 
required by law in about a fifth of the states. In the states 
where the assessment data are being used to make state-level de- 
cisions, samples rather than all students are being assessed, v/hile 
In the local-level, decision-making states all students in the 
target populations are being assessed. Criterion-referenced 
Instruments are very common with the states where the data are 
being used for state- level decisions, but the states collecting 
Information for local decision making are favoring norm-referenced 
instruments. Finally, no dominant funding pattern has evolved in 
either of the two groups of states. 2? 

State Adaptations Qf, Model 

In the assessment of Citizenship education, Maine made an 
extensive application of the National Assessment model and carefully 

^bid .^ p. 1. 
^ ^Ibtd .y pp. 2-3. 
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duplicated it so that comparable data were collected at the state 
level.*' Maine's first cycle of the ten subject-matter areas of 
National Assessment is to be completed by scheduling two of these 
areas each year for five years. Citizenship and Writing \vere the 
first subject areas to be assessed. 

Based on the results of a previous study of objectives for 
education in Maine, two review committees decided to accept the 
National Assessment objectives as being closely related to the 
Maine objectives. Maine selected the 17-year-old population of 
in-school students for its first assessment. A sample of 2,000 
17-year-old students was used to represent the approximately 
17,000 17-year-old students in the State. The State was divided 
into four geographical regions. As in National Assessment, school 
buildings were randomly selected from the geographic regions, and 
students were then randomly selected from the buildings. Packages 
were developed with exercises taken from the two subject areas. 
The available, released exercises from National Assessment were 
carefully examined to see if they reflected objectives valid for 
Maine and to see if some could be modified, where needed, to be 
administered in group sessions using the paced-tape method while 
still retaining a high degree of comparability to the National 
Assessment individually administered exercises. The packages were 
made up of 23 Citizenship and seven Writing exercises, plus a 23- 
item Student Questlorinaire. The exercise format was kept virtually 
identical to the one used in National Assessment. Trained 

^ ^Maine Assessment of E ducational Progress? Methodology (Report 5), 
Department of Educational and Cultural f?ervices, Augusta, 
Maine, 1972. 

o 
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administrators were sent out to administer the exercises, and the 
exercises were scored according to National Assessment procedures • 
On data reporting and analysis, there was the census-like reporting 
of the performance of the Maine students plus comparisons of the 
Maine results with appropriate National Assessment data. 

In summary, the Maine Assessment duplicated the National Assess- 
ment procedure as completely ajs possible. With minor exceptions, 
the same objectives were used for Citizenship, The same sampling 
design was used with adaptations to a smaller geographical area 
and population. The exercises were for the most part taken from 
those released by National Assessment, and they were organized in- 
to packages similar to those used by National Assessment. The ad- 
ministration and scoring of the exercises were conducted in the 
same manner as National Assessment worked. Since the same private 
contractors were used by Maine as were used by National Assessment, 
the duplication was complete wherever possible. The reporting and 
data analysis were similar, and the data did provide the opportunity 
to compare the results in Maine with the results from National 
Assessment. 

Here, the model was very carefully duplicated at the state 
level. The big question which comes to mind after studying the 
Maine Citizenship report is, "Aren*t the National Assessment data 
being treated here as some kind of a national norm against which 
the performances of 17-year-old students in Maine were being com- 
pared?" Of course, this use of National Assessment data had been 
questioned from the start of the proposal for an assessment at the 
national level. Now, Maine has provided the opportunity to study 
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the effects of this use of the data on the educational system of 
a state. 

Another state which carefully followed the model was 
Connecticut,^^ Here, an assessment was first conducted in Reading. 
To permit comparisons, the Connecticut program used available in- 
struments and applicable procedures developed by National Assess- 
ment which were adapted to the requirements of the local situation. 
Connecticut's Reading objectives were matched to the Reading ob- 
jectives of National Assessment. Approximately 220 reading exer- 
cises from National Assessment were used in producing the packages 
used in the Connecticut assessment. Exercises were selected to 
represent all of Connecticut's Reading objectives. The age groups 
assessed were 9, 13 » and 17. As with the National Assessment 
packages, tape recorded instructions were used. The sampling 
design was a multi-staged design duplicating with few exceptions 
the National Assessment design. As with National Assessment, a 
group of administrators for the packages was recruited and trained. 

This was another example of careful duplication of the National 
Assessment model down to using the same objectives and exercises. 
Again, there was the use of the National Assessment results as 
norms to which the Connecticut results were compared. 

The-Texas Needs Assessment used the model for the development 
of their assessment in mathematics at the sixth-grade level. 

^ ^Report oa JihS Assessment a£ Reading Skills j^f Connecticut publi^c 
School Students y Institute for the Study of Inauiring Systems, 
Philadelphia, Pa., and Department of Education, Hartford, 
Conn., 1972. 

^ ^SjLxth - Grade Mathematics ? A Needs Assessment? . Report , Texas Edu- 
cation Agency, Austin, Texas, 1972. 
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However, while using ideas from the model, they broke with It In 
a number of places. The Texas people were concerned that tho 
assessment would yield Information which would be useful to teach- 
ers^ in their classroom instruction of students* Prom a pilot 
stuay, it was decided to use a criterion-referenced reading test 
and to work with grade levels Instead of age groups of students. 
They worked with the sixth grade, and the tests were administered 
by the staff of each school which participated in the assessment. 
The objectives were chosen from the major skill areas treated in 
the state-adopted textbooks. Regional location and community size 
were taken into consideration in selecting the sample « Approxi- 
mately 10 per cent of the Texas schools teaching at the sixth-grade 
level administered tests, and approximately 10 per cent of the 
pupils being taught at the sixth-grade level were included in the 
sample. Reports were given to teachers on the performance of their 
Individual students. Also, there was a school report on the per- 
formance of the students for each school and a report on each of 
the classes in the school. Comparisons were made on the basis of 
sex, race, and size of community. 

The Colorado Needs Assessment, while using the model, made an 
even greater break with It.^^ Its objectives were based on a 
state study of educational goals, and the educational goals were 
restated in terms of performance objectives. Following the model, 
objective-referenced exercises were written. A sampling design 
was used and the student responses were analyzed. In this assess- 
ment, classroom teachers were Involved in the writing and refinement 

^^Helper, John V/., Aq Aasesatn e.ii_t jqI Learn er Needs. JLq Col?r,^^.Q, 
Colorado Department of Education, Denver, Colo., 1972. 
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of the behavioral objectives. Objective-referenced exercises 

were written for nine subject areas. The exercises were administered 

to a sample of 30,000 Colorado students. A stratified random 

.--.sampling procedure was used to select a sample of school districts 
of .iie State. Then schools were selected at random from the dls- 

-rtrlcts chosen. Finally, classes in school buildings were randomly 

-chosen for testing. The samples were representative of all Colorado 
students in grades 3, 6, 9, and 12. A group of proctors was hired 

-• and trained to administer the exercises, and the exercises were 

^.scored by computer. The data were analyzed on a statewide and 
.district basis, and the results were broken down by subgroups, 

,.e,g. boys, girls, urban, rural. 

..As pointed out earlier, states are rapidly moving into the 

5 : assessment field. Some are reproducing the National Assessment 

• - model at the state level, and others are developing variations of 
*. -the model. The more crude efforts have resulted in endless pages 
i:' of raw percentages without any explanation of the results. Based 

i:. on a survey of state assessment programs. Beers and Campbell identi* 
r:^ied several of the problems which are common to these state 
rrprograms.^^ Naturally, a shortage of money and staff were the 
?'-most frequently mentioned problems, for it is a fact that many 
rtstates have moved into this area without providing adequate funds 
f'jfor a realistic assessment program. Also, teacher resistance to 
(vcassessment and negative public attitude toward outside testing 
w-.!Mere problems mentioned. Test results have been misused in the 
p&past, such as the firing of teachers based on Incorrect interpre- 

• tatlon of test results. Also, test developers have been guilty 
c: of. violating the privacy of students through questions which 

•^^ State Educatlo . r^al Assessment, programs ^ Beers and Campbell, 
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transgressed the examinee's human and l?»gal rights. A third 
problem area has been with the utilization and dissemination of 
results. fJome school officials do not understand the results. In 
some situations, there has been hostility to the results. Some 
officials have ignored results In making decisions. Finally, re- 
sults have frequently not reached the right people In a useable 
form. 

jy[S£ JSdC JiM ^todel ^ its. PA,gtrl<?t hS^Sl 

To date, there have been a limited number of efforts reported 
on the use of the National Assessment model at the district level. 
Three such assessments on which some data have been released are 
being conducted in Lincoln, Nebraska; San Bernardino, California; 
and Montgomery County, Maryland. 

In Lincoln, Nebraska, the exercises released by National 

Assessment in Citizenship and Writing were used in a local assess- 

31 32 

ment which yielded data comparable to National Assessment data. ' 
A group of supervisory personnel from the central office Identified 
the Citizenship objectives which were applicable to the Lincoln 
schools. Then the released National Assessment exercises were 
selected which were applicable to these Lincoln objectives. Also, 
the National Assessment model was followed in selecting a random 
sample of 13 year olds from the Lincoln junior high schools. In 
addition, a sample of in-school, 17 year olds was tested on some 

3^n\^eekly Focus," Lincoln Public Schools, Lincoln, Nebraska, 
February 12-19, 1973, p. 3. 

32Brandt, Ronald, Associate Superintendent for Instruction. 

Report on Assessment Results to %ard of Education, Lincoln 
Public Schools, Lincoln, Nebraska, Spring, 1973 • 
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of the writing exercises. The administration of the exercises 
was carried out by a group of specially trained administrators, 
and the tape-paced method was used in presenting the exercises to 
the students. Scoring followed the National Assessment procedures, 
and in reporting the results comparisons were made to National 
Assessment data with special attention given to comparable sub- 
groups such as cities of similar size and the same geographic 
region. 

The San Bernardino City Schools developed a criterion-referenced 
assessment model of student progress which was based on the National 
Assessment model. This model involved local teachers, students, 
and laymen in a good setting; eight educational goals were identi- 
fied through the efforts of workshops involving teachers, students, 
and patrons. A Curriculum Task Force composed of 20 teachers wrote 
behavioral objectives for the goals to be appropriate for grades 
3, 5, 9, and 12. National Assessment consultants assisted the 
teachers in developing exercises to assess the stated objectives 
at these grade levels. Also, the National Assessment consultants 
helped to design a sampling procedure to provide district-wide 
representation. The exercises were organized into test batteries 
for each grade level. The Teacher Task Force administered and 
scored the tests. This is a break from the National Assessment 

%onney, Lewis A., ••Application of the National Assessment of 
Educational Progress Philosophy in San Bernardino City Uni- 
fied School District, •' Unpublished Paper, San Bernardino 
City Unified School District, San Bernardino, California. 

3^Speclal Curriculum Task Force, "Report on Student Performance," 
Office of Instructional Services and Research and Develop- 
ment Office, San Bernardino City Unified School District, 
San Bernardino, California, June, 1972. 
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practice of using specially trained exercise administrators. The 
results were tabulated in terras of percentage of students meeting 
stated behavioral objectives. While the National Assessment model 
was followed in many ways, such as use of behavioral objectives, 
criterion-referenced assessment instruments, and sampling of target 
populations, the assessment was designed for application at the 
local level, and it was planned, developed, and carried out by 
local personnel. 

The Montgomery County Schools, Maryland, developed a program 
for assessing 13- and 17- year-old students. In this assessment, 
the released National Assessment exercises for Writing were used. 
These were administered in two group-package sessions to samples 
of 13- and 17-year-old students. The results for Montgomery County 
students were compared to the results from the nationwide samplings 
of 13 and 17 year olds by National Assessment. One of the varla- 
tloiis in the Montgomery County sampling design was stratification 
by I.Q* and grade level. The purpose was to spread the sample across 
the grade by school-I.Q. groups; however, these sampling groups were 
not us9d as reporting units. Bach age group (13 and 17) was strati- 
fied by I.Q. groups (low and non low) and by grade levels. 

Adaptation o£ ^ Mp^el 

In the above discussion, it is evident that there will be as 
many adaptations of the model as there are local and state units 

■ I'lii J Ill I ill I ■ ii w ■ n iiiiii<<ii 11 II ■'■ ■■It' I ' II nil I ■ '■III II II ml 

^^Bayless, David L., Ralph B. Folsom, and Louise H. Lewis, "Sample 
Design for Assessing Montgomery County Public Schools 13- 
and 17-year-Old Pupils Using the NAEP Model," National 
Assessment of Educational Progress and Educational Commission 
of the {^itates, Denver, Colorado, January, 1973* 
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conducting assessments. Probably, there Is no specific assessment 
model which is the best; hence, there is no model that should be 
applied without modification in an3' and all situations. Neverthe- 
less, there are principles of good assessment which should be 
applied in developing or adapting a model for local assessment 

purposes. Listed below are some characteristics which should be 
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found in a good assessment program.-^ 

1. The program has clearly defined goals that apply to a 
particular audience or audiences. 

2. The program has a realistic number of goals which are 
attainable under the existing assessing conditions. 

3. The program has established priorities among its goals 
and places its major efforts on its major goals. 

if. The program has been designed to gather information 
considered to be important in education. 

5. The program has specific objectives which it is striving 
to attain. 

6. The program has been designed to provide results at a 
useable level of accuracy. 

7. The progran has used data-gathering instruments which 
measure the objectives of the assessment. 

8. The program has collected data in such a manner as to 
introduce a minimum of error in the results. 

9. The program has scored and processed data in an accurate 
manner. 

10. The program has used analytic techniques that provide 
the data breakdowns needed by decision makers. 

11. The program has reported results in a manner useable by 
its audience. 

12. The program has provided help in the interpretation of 
results and assistance in their implementation. 

1^. The program has provided for the active Involvement of 
groups of persons from all of the major audiences for the assessment 
results. 



S^wnmpT*. P^vflnw B. . Developing ^ Lajtisfi ^s^a i ? . r , g^ram , 

Cooperative Accountability Project, Denver, 1973» p. 89. 
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IMPLICATIONS OP THE MODEL FOR CURRICULUM DEVELOPMENT • 

Of course, a major, potential outcome of National Assessment 
and the model was the providing of new, accurate data with regard 
to curriculum problems. Curriculum decision makers are furnished 
data which have not been available to them before this. Because 
of this new information, they should gain new insights into their 
problems, and hopefully there will be innovative aporoaches taken 
to the solution of these problems resulting from this.^^ 

Implic a tions q1 ^ Model 

The assessment model has potential for promoting curriculum / 
development. This is especially true when it is applied to state 
or local situations in the manner used in Colorado and San Bernardino. 
In these two situations, objectives were developed which specifi- 
cally applied to the local situation. The statement of well written 
objectifies in behavioral terms may sharpen the purposes of instruc- 
tion. Through the experience of writing behavioral objectives, the 
curriculum worker gains a much clearer perception of his task*, hence, 
this practice may have a beneficial impact on curriculum work. On 
the other hand, the use of behavioral objectives has not always been 
a positive influence. The objectives may zero in on easily defined 
behaviors which lack scope and significance. They may produce 
tunnel vision, and put stress on the inconsequential and trivial. 
In an effort to be specific and to define the exact behaviors desired, 
the larger perspective may be lost. 



^^Womer, Frank B. and Marjorie M-Mastie, "How Will Natio^^ 

Assessment Change American Education?" £M MM MmD 53s 
118-20, October, 1971. 
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Again, the development of exercises from the Identified be- 
havioral ob.1ectlves may have a positive Influence on curriculum. 
The kind of new, innovative exercises which have been developed 
by National Assessment may have a very positive influence on what 
is being taught and how it is being taught. Teachers both in re- 
.vlewlng exercises which have been used in National Assessment and 
In writing exercises for local assessments may be influenced in 
their selection of both content and methods by their knowledge of 
-these assessment exercises. Material not relevant to the objectives 
of the course m.ay be dropped, and methodologies promoting the kind 
of skills needed in the assessment exercises may be introduced. 

On the other hand, the results may be less desirable. If 
•In local and state situations the dictates of finances or the lack 
.of leadership result in the use of poorly written, machine- scored, 
multiple-choice exercises, the results may be very negative. 
•Teachers may feel pressured to stress rote learning of facts in 
•order to prepare their students for poorly written examinations. 
cHence, poorly written exercises may keep irrelevant material in 
'the curriculum and limit curriculum innovation and development. 
!-.The quality of the exercises written and released will have an 
!:lrapact on curriculum development. 

iGood sampling procedures may give insight into the status of 
rknbwledge, understandings, skills, and attitudes of students in 
a-particular target population. This can promote curriculum im- 
provement and innovation. Problem areas in the curriculum may be 
cldentified. From the National Assessment, there have been some 
problem areas identified in the Citizenship results. On an exercise 
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dealing with freedom of speech, a large percentage of 13, 17, and 
adult age groups Indicated that they would not allow sample con- 
troversial statements to be made on radio or TV,^^ This showed a 
lack of understanding or valuing of the Constitutional right of 
freedom to experss controversial or unpopular opinions. 

The results on the Citizenship assessment indicated that 
black, urban students in our large cities compared poorly on 
knowledge about the structure and function of government to the 
national average performance on the same exercises. 39 

On the other hand, there are potential difficulties with 
assessment data which represent national levels of performance. 
Even though the data were not collected with this intention and 
were reported in census-like form, the results of National Assess- 
ment are being treated like national norms. Several states have 
conducted their own assessments duplicating the National Assess- 
ment model so that they can make direct comparisons between their 
state results and the various national, regional, and subgroup 
results. There is the potential of great mischief in this approach, 
for- it may lead to unfair comparisons between groups, states, and 
regions. In the assessment reports of some states, tables of per- 
centages have been presented without any interpretation or expla- 
nation; Some school systems have been presented in a very bad way 
without any reference being made to the kinds of variables involved 
in: the different learninti; situations. Such variables as per pupil 

3Bcampbell, Report 2, Cl^ izenshiP i National Results.^ 
p. 3^. 

B^Norris, jlI, National Assessmen t Report 2, op, cit., 
pp. 63-65. 
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expenditures, educational level of parents, and motivation of 
pupils do have an Impact on the learning situation. These and 
other variables cannot be ignored In interpreting the results of 
assessment* 

Here, it is not being suggested that assessments sho'ild not 
be conducted because there are potential misuses of the data, but 
it is being pointed out that misuses of data do occur. This does 
mean that those engaged in assessment at national, state, and local 
levels have the responsibility to be constantly engaged in an edu- 
cational program to aid those using the data to make correct inter- 
pretations of it. We need these kinds of information for decision 
making, but if the data are misused or misinterpreted, then the 
decisions based on them may not be good ones. 

I j^pact m Curricul um 

National Assessment is providing data on which decisions can 
be made. The reports on both Science and Citizenship have resulted 
in strong recommendations for curriculum changes in these subject 
areas .^^ As data are gathered at the state level throufrh the use 
of the model and its adaptations, specific suggestions for changes 
may be made. For example, Texas has designed its state assessment 
so that there is direct feedback at the classroom level. This may 
have a strong and immediate impact on these classro^^m?; which could 
be either good or bad depending on what interpretations and recom- 
mendations are made. 

^^Harrlson, Charles H., "Are We Educating for Tomorrow?" 
Scholasttc Teacher, f September 21, 1970, pp. 16-19. 
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There are other problems which should be pointed out con- 
cerning curriculum decision making and national and state assess- 
ment results, i^ile the National Assessment process for identify- 
ing objectives provides for broad-based participation in the 
decision-making groups, objectives still are selected which neglect 
sizeable, subcultural groups in our nation. This means that even 
some of the very general objectives which are selected may not 
apply to these groups, and thus they are not valid for some sltua- 
tions« 

To illustrate the potential difficulty with objectives and 

subgroups, consider Objective V of the Social Studies. This is, 

"Have a reasoned commitment to the values that sustain a free 

society." This has under it subpoint B, "Believe in the rule of 

law and can justify their belief." For blacks in the core city, 

this objective may not carry value. Their experiences may have 

been that the law is used against them, and they need to constantly 

^2 

be looking for new ways to work around the law. \7hile this is 
perhaps an extreme example, it does make the point that in as 
diverse a population as we have in the United States, there will 
be many subgroups for whom the broad general objectives do not 
carry the same meaning as for other members of the nation. 

There is the question of what are the components of a general 
education or what should be the components. The selecting of 



^^Wrightstone , J. Wayne, Thomas P. Hot^an, and Muriel M. Abbott, 
"Accountability in Education and Measurement Problems," 
ye.s.t, Service N otebook 33, Hare our t. Brace, Javanovtch, Inc., 
New York, p. 

^^Norris and Goodwin, Social Studies Objectives, op. cit., p. 26. 
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common objectives for a subject area such as Science and the writ- 
ing of exercises for these objectives provide one definition of 
^at students of a certain age group are expected to know about 
science. Since this establishes what comprises general education 
in science, it has a definite limiting impact on the freedom of 
choice of a student in deciding what he wants to learn. He is being 
dictated to concerning what he should take in general education. 
This situation always has existed in education, but it puts the 
assessment movement in the camp of the conservatives in the current 
controversy with respect to free choice and unlimited electives for 
students. Here, the assessment movement is counter to the humaniz- 
ing movement in American education.^^ It is promoting a closed 
rather than an open approach to curriculum. 

The very Identification of a subject area for National Assess- 
ment will have an impact on the fortunes of that subject area. The 
"chosen" subject areas are more likely to be supported financially 
and retained in the curriculum than those not selected by National 
Assessment. Those subject areas which are not included in the 
"magic" ten may find that they are second class so far as school 
boards are concerned. If the students of the district have made 
a -poor showing on the state assessment, funds may be shifted to 
the. support of those subjects where the low scores were identified. 
The subject areas which are not getting this public exposure may 
find their financial support reduced, t^/hile the potential for 
better educational decision making is here, there is also the 

^3Hencley, Stephen P.. "Impediments to Accountability," Admlnj* 

stra tor's Note book ^ Vol. XX, University of Chicago, December, 
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potential for poorer educational decision making because of the 
inpact of public exposure of the asf^essment results on the decision 
makers* 

The foreign languages are not Included in the ten subject 
areas being assessed • The fact that they are omitted means that 
no assessment Is being made of our foreign language skills. Lan« 
guage departments will not be able to cite assessment data as evl* 
dence that there Is need for greater support of their programs. 
They have been put Into a poorer bargaining position by this omis- 
sion* 

So far as this writer knows, no assessment Instrument of this 
type has been produced to date, but the potential for "shortcut" 
assessment schemes by publishers is certainly there, V/hy go to all 
of the work for an expensive local assessment effort such as San 
Bernardino's when you can buy a commercial assessment package which 
Is based on the released National Assessment exercises and which 
will provide the school district with results that may be compared 
to National Assessment results (norms). The exercises may be given 
and scored by the local teachers, and the results may be compared 
by them with the National Assessment results which have been In- 
cluded with the commercial assessment package. If the project is 
handled "right," a "llvewlre" superintendent can demonstrate that 
his district is outscorlng the National Assessment results (norms) 
at a very modest cost* 

Before leaving this idea of a commercial assessment package 
based on released National Assessment exercises, the positive 
potential of this idea should be mentioned. With honesty and 



careful application, this approach could be of considerable value 
to school districts. First, the objectives being assessed by th« 
exercises included in the instrument need to be carefully identi- 
fied. If the school district finds that these objectives fit their 
own objectives adequately, the exercises may be used with confidence. 
Second, there is no reason why teachers should not be able to con- 
duct and score the exercises accurately. A tape-paced administra- 
tion could be used, and the directions for scoring could be written 
in such a way that teachers could follow them with good results. If 
a district collects data about the performance of its students on 
National Assessment exercises, these could be very valuable data. 
This would be worthwhile if these data could be collected at a mod- 
est cost. There are a good many "ifs" and pitfalls in this proposal, 
but it is true that accurate, valid data could be collected in this 
fashion at a reasonable cost for a school district. Undoubtedly, 
some districts will find this to be a workable plan for them. 

Cost is one of the big problems. It took a great deal of money 
to develop the highly sophisticated program of National Assessment. 
Currently, the data being provided by National Assessment are of 
the highest quality, and the results have great promise for pro- 
moting educational improvements. Nevertheless, hundreds of local 
districts are considering their own assessments, and the question 
of how to reduce the cost is becoming important. Should they develop 
their own assessment instruments, or should they use a commercial 
version of the model? For many districts, it is not possible for 
them to carry out their ov/n assessment program; hence, for many 
reasons, the second alternative will be used. Over the" coming years, 
the commercial assessment package will probably become reality. 
The only ott:er alternative open to poorly financed districts 
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Is to resist the pressure to become involved in local assessment. 

One of the potential outcomes of an assessment program is the 
spinoff of research projects designed to investigate questions of 
causality that are raised by assessment results. Several places 
In this paper illustrations have been given where further investi- 
gation is needed in order to more accurately interpret results. 
There are any number of potential doctoral studies in the data 
coming from National Assessment which need to be made before the 
results may be confidently used in curriculum work. Currently, 
one of the barriers to conducting a doctoral study of this type 
Is the security measures which National Assessment of Educational 
Progress believes that it must take to safeguard Its operation. 
At present, the researcher finds himself being confronted with the 
need for Information which he must have to proceed with his study 
but which is not made available to him. The results are that it 
is very difficult to conduct these needed studies. It is suggested 
here that the policies which govern the use of National Assessment 
data in these kinds of studies need a thorough review at this time. 
No one is disputing the need for security in order to assure un- 
contamlnated results, but the issue of overcaution is being raised 
by this writer. These spinoff kinds of studies have the potential 
of making an Important contribution to the overall effort Ir. 
assessment, and they should not be killed off by bureaucratic kinds 
of decisions. 

There is a final interesting prospect in this entire assess- 
ment business, and that is the Increased pace of change in our 
society within recent decades. Those of us who have been involved In 
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educational decisions over the last couple of decades Know that 
catching the direction of things is frequently more important than 
trying to make careful, data-based decisions. By the time we have 
collected and analyzed our data base, society and technology have 
gone off and loft us. It is hard to fault the soundness of the 
data collection design of National Assessment, but there is a five- 
year time lapse from start to finish of a cycle plus the time lag 
of scoring and data analysis. The question is whether this is 
the best process for educational decision making in the last quarter 
of this Century. Are changes in our society coming so fast that 
long before the data base is established the data are no longer 
relevant for the decisions for which they were gathered? 

Assessment data are badly needed, for «;hey point to the 
problems and inequalities of our educational system which require 
attention.. The promise of National Assessment and of the adapta- 
tions of the model to state and local assessments are great, and 
the evidence supports moving ahead with this valuable project. 

USB OF THE MODEL FOR ACCOUNTABILITY PURPOSES 

Assessment is not the same thing as accountability, for ac- 
countability places greater emphasis on value judgment than assess- 
ment does.^^ It is concerned with the badness or goodness of some- 
thing. Educational assessment is aimed at improving educational 
decision making by collecting information concerned with the out- 
comes of education. Accountability has varying meanings depending 
on who is writing about it. 

A number of approaches have been proposed to make schools more 
accountable, such as systems approach, management by objectives, 

p. 3* 
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education program auditing, planning-programming-budgeting system, 
performance contracting, voucher plan, and alternative educational 
forms. A widely accepted interpretation is that accountability ^ 
is to determine if the teacher who is assigned the task of educating 
a group of students is performing that task. On the other hand, 
a broader interpretation of accountability is that it is a pro- 
cess for determining if the program of a school, district, or 
state is producing the student achievement expected with regard 
to the objectives of the program.^'' The first of these interpreta- 
tions puts the responsibility for individual pupil achievement on 
the teacher. The second places the responsibility for the output 
of a program of instruction on the school, district, or state school 
system. 

The first interpretation of accountability has been widely 
debated and has gained the opposition of many groups including y/ 
the teachers* organizations. A model of this approach to account- 
ability is given below. 

'^^Browder, Lesley H,, Jr.. JLa A<itninistrator 's Handbook oq E^- 
catlonal Accountability , American Association of School 
Administrators, Arlington, Virginia, 1973, PP. 1^-21. 

^Wrlghtstone , Kogan and Abbott, ail., p. !• 

^7Gronlund, Norman B., (Ed.), Readings JLn Meflsurement Evfil-^ ii 
jaajtistQ, Lee J. Cronbach, "Evaluation for Course Improvement, 
Macralllan Company, New York, 1968, p. ^1. 
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ACCOUNTABILITY MODBL^^ 

I. Behavioral Objectives 
II. Stated Evaluation Criteria Related to the Objectives 

III. Scheduled Materials, Learning Activities, Equipment, 
Etc. 

IV. Teaching Activities 

V. Evaluation of Student Performance Based on Behavioral 
Objectives 

Hot only does this model stress the output of the teacher in terms 
of pupil achiaveraent, but it does not provide for the multitude of 
variables which are found in any teaching situation. The primary 
concern in the model is, "Are the children learning what the teacher 
vas hired to teach them?" No allowances are made for the many 
•variables such as pupil ability, parental education, or wealth of 
school district, which may influence the success of the teaching 
In a given learning situation. 

Under this approach to accountability, complete data must be 
collected for each individual in the population, for here it is 
necessary to identify the performance oT specific individuals 
vith respect to stated objectives since the responsibility for the 
Individual student's performance is to be assigned to the instructor 
•ifho was charged with teaching for these objectives. Here, it would 
seem that accountability is taking us back to yearly, mass testing 
-with vhich some of us are only too familiar. While it is hoped 
that some of the improvements in test administration such as tape- 
paced administration introduced by the National Assessment model 
vould be used, it is suspected that because of financial cons,ider- 
ations, teachers would be involved in the administration of t hese 
^^Browder, jgja. P» 
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tests with many of the problems concerning results which have 
been true for this in the past. 

The second interpretation of accountability presented here 
stresses the performance of a given group of students, and it is . 
concerned with what the strengths and weaknesses of a program are, 
as well as where students are or are not achieving. The National 
Assessment model may easily be adapted for this type of account- 
ability use. In assessment, the objectives are identified, the 
achievement level with respect to these objectives in a popula- 
tion Is determined through a sampling assessment procedure, and 
the results are reported in terras of what percentages of the popu- 
lation are achieving the objectives. 

The National Assessment model, as it was originally designed, 
did not provide useable data for determining who was responsible 
for the individual student either achieving or not achieving the 
stated objectives. As designed, the model provided information 
concerning the achievement on the stated objectives of a population 
or -subgroups of that population; hence. It was not possible to 
identify the results for individuals in these population groups. 
Also, it was not possible to establish what individual teacher 
was responsible for the students having either achieved or not 
achieved the stated objectives. 

Now, let us examine the National Assessment model for its 
application to the evaluation of an entire curriculum for account- 
ability purposes. The model components identified in this paper 
are; ^ objectives development, exercises development, sampling plan, 
administration of exercises, scoring and analysis, reporting and 
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dissemination, and utilization. Under accountability, objectives 
may be developed with input from a number of sources including 
the patrons who are paying the bill for education, or an already 
established set of objectives may be used.^^ Since this is the 
evaluation of a given curriculum, the objectives should apply to 
that program if the results are to be valid. Who should select 
these objectives is an issue in accountability. Shouldn't there 
be input here from the teachers who are presenting the program? 

While there are some problems with respect to how objectives 
are selected, the components of the National Assessment model are 
all functional in this adaptation of it for accountability pur- 
poses. Measurement specialists contend that objective-referenced 
exercises which are written for specific learning objectives are 
better for use in accountability evaluation than the norm-referenced 
exercises commonly used in standardized tests. Therefore, re- 
leased objective-referenced, National Assessment exercises may 
be used for accountability purposes where their objectives are 
valid for the learning situation which is being evaluated. 

In the National Assessment design, the samples were drawn in 
such a fashion that they represented the population of 9, 13» 17, 
and 26-35 year olds and subgroups from these populations. Where 
the model is being used to evaluate the performance of a given 
population, sampling procedures may be used as they were developed 
for the National Assessment model. Also, the same professional 
care which is taken in exercise administration of the National 

^^Morrissett. Irving, "Accountability, Needs Assessment and Social 
Studies," SoGlai Education 37$ 37^, April, 1973. 

^^Wrlghtstone, ^jj., p. 5» 
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Assessment model would be desirable In the application of the 
model for accountability purposes. Scoring and reporting are 
the same for assessment and accountability, and the same percentage 
kind of reporting can be used in both situations. 

Where accountability is being applied to a total organiza- 
tion such as a school, a district, or a state, the National Assess- 
ment model may be used with little or no modification. It was 
designed to accurately establish what the level of performance on 
a given set of objectives wais in a population, and it c^n be used 
to do this for accountability purposes as well as assessment pur- 
poses. Likewise, it can assess subgroups of the population and 
Identify specific strengths or weaknesses in the performance of 
a given subgroup. The model is an excellent instrument for carry- 
ing out accountability in thl« kind of situation. 

There are many value questions related to accountability, and 
while it is not the task of this paper to discuss them, a number 
of them are cited here. Will tight state accountability structures 
severely limit creativity and innovation in the schools? As indi- 
cated in some of the previous comments, doesn't accountability have 
the potential of becoming a stulifying state testing program? Will 
the single-minded pursuit of achieving performance objectives lead 
to the abuse of children? Couldn't a disproportionate amount of 
time be expended on the defining of objectives without a commen- 
surate increase in learning? Could not objectives established by 
politically oriented groups set schools on a course of indoctrin- 
ation? Currently, in education there is a humanlst-behaviorist 
conflict. Doesn't the accountability movement support a kind of 
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teohno-urban fascism? Could not the accountability raovement 
lead to the teaching of easily quantifiable material and discourage 
the inclusion of material which is difficult to quantify? Doesn't 
accountability promote a closed rather than an open educational 
system? Doesn't accountability present obstacles to the continued 
development of freedom and autonomy for teachers? This is hot an 
exhaustive list, but these are all disturbing questions. Further- 
more, there is considerable evidence in the literature which sup- 
ports the seriousness of the Issues being raised by these questions. 

SUMMARY 

In closing, there are several summary statements which can 
be made. The National Assessment model Is no doubt the best and 
most comprehensive procedure that has been designed for collecting 
data for these purposes. The model can and has been successfully 
adapted for use at the state and local levels, and as states become 
more active in assessment, there will be many adaptations of It made 
to fit local needs. Finally, where the concern is with the evaluation 

5lHencley, op. clt. 

52Gubser, M. M,, "Accountability As a Smoke Screen for Political 
Indoctrination in Arl25ona," PM Delta Kappan 55? 6^4-5, 
September, 1973« 

^^Hand, Harold "National Assessment Viewed As the Camel's 
Nose," £M Delta Kapoan U7sfi-13, September, 1965. 

5^House, Ernest R», "The Price of Productivity: \'lho Pays?" 
Today's. E ^upation 62:65-9, September, 1973« 

55Landers, Jacob, •'Accountability and Progress by Nomenclature: 
Old ideas in New Bottles," Phi Delta Kapnan 5^:539-^1, April, 

.1973. 

9^0rnsteln, Allan 0. and Harriet Talmage. "The Rhetoric and the 
Realities of Accountability," Today's Bducatlon 62:70-80, 
September, 1973. 



of group perf rtrmance, the model may be used for accountability 
purposes. With the current rapid development of state assessment 
and accountability programs, it Is expected that the model will 
be widely used for these purposes in the coming years. 
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